40-A136  469  DIFFUSION  APPROXIMATION  IN  ARROW'S  MODEL  OF  EXHAUSTABLE  1 // 
RESOURCES(U)  STANFORD  UNIV  CA  INST  FOR  MATHEMATICAL 
STUDIES  IN  THE  SOCIAL  SCIENCES  D  DUF FIE  ET  AL .  AUG  83 
UNCLASSIFIED  TR-416  N00014-79-C-0685  F/G  12/1  NL 


MICROCOPY  RESOLUTION  TEST  CHART 

NATIONAL  BUREAU  Of  SIANDARDS-I963-A 


A 1 36469 


•'Wit, 


DIFFUSION  APPROXIMATION  IN  ARROW’S  MODEL  OF 
EXHAUSTABLE  RESOURCES 


Darrell  Duffle  and  Michael  Taksar 


Technical  Report  No.  ^i6 
August  1983 


Jjlco®33ion  For" 

»XIS  CRAM 
OTIC  TAB 

Unannounced 
Justificatl 


or 


By-  ___ 

f.Wrtrtfcntlon/ 
Availability  Codes' 
Avail  and/op 
Special 


r 


A  REPORT  OF  THE 

CENTER  FOR  RESEARCH  ON  ORGANIZATIONAL  EFFICIENCY 
STANFORD  UNIVERSITY 


Contract  ONR-N00014-79-C-0685 ,  United  States  Office  of  Naval  Research 


THE  ECONOMICS  SERIES 

INSTITUTE  FOR  MATHEMATICAL  STUDIES  IN  THE  SOCIAL  SCIENCES 
Fourth  Floor,  Encina  Hall 
Stanford  University 
Stanford,  California 
9^305 


□O 


DIFFUSION  APPROXIMATION  IN  ARROW'S  MODEL  OF  EXHAUSTABLE  RESOURCES* 

by 

Darrell  Duffie  and  Michael  Taksar 


1. 


Introduction 


71. 


7 


->  What  we  shall  call  ""Arrow's  Model"  for  resource  exploration  and 
use  under  uncertainty  was  first  formulated  by  Arrow  [l977l  and  analyzed 
by  Arrow  and  Chang  [1978] .  This  model  proved  to  be  very  successful  and 
stimulated  a  lot  of  research. \The  most  interesting  further  analyses  of 
Arrow's  model  and  some  numerical  results  have  since  been  produced  by 
Hagan,  Caflisch,  and  Keller  [1980 ]  (henceforth  "H-C-K"),  and  indepen¬ 
dently  by  Derzko  and  Sethi  [l98l] .  The  purpose  of  this  paper  is  to 
bring  to  a  "common  denominator^the  results  of  previous  studies  and  to 
introduce  a  new  diffusion  model  approach  to  the  same  problem.  The 

problem  is  posed  roughly  as  follows.  -  - 

A  central  planner  charged  with  the  exploration  and  use  of  some 
natural  resource  (for  example,  oil)  has  knowledge  of  the  current  stock¬ 
pile  of  the  resource  (R  >  0),  the  remaining  unexplored  land  (A  >  0), 
and  the  probability  distribution  of  undiscovered  resources  over  the 
remaining  land.  For  simplicity  it  is  assumed  that  the  random  quantities 
of  the  resource  to  be  extracted  from  any  two  disjoint  sections  of  land 
of  equal  area  are  independently  and  identically  distributed.  Then  any 
exploration  policy  is  merely  a  specification  for  a  decreasing  process 


•This  research  was  supported  by  the  Office  of  Naval  Research  Grant  0NR- 
N0001U-79-C-0685  at  the  Institute  for  Mathematical  Studies  in  the  Social 
Sciences,  Stanford  University. 
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A  =  {A(t);  t  >  0},  where  A(t)  is  the  amount  of  unexplored  land  remain¬ 
ing  at  time  t  when  policy  A  is  followed.  We  define  N^(t)  to  be 
the  (random)  cumulative  amount  of  resource  extractd  by  time  t  when  the 
policy  A  is  followed. 

The  econony  has  utility  at  the  rate  u(c)  for  consuming  the 
resource  at  the  rate  c  >  0,  where  u(»)  is  differentiable,  increasing, 
and  strictly  concave.  With  time  discounting  at  the  rate  p  >  0,  the 
total  utility  for  a  particular  path  of  consumption  {c  ,t  >  0}  is 
then: 


/e“ptu(c  )dt  . 

0  x 

Exploration  requires  inputs  of  "goods  in  general"  at  a  constant  ratio  of 
P  >  0  per  unit  of  land  explored.  Arrow  and  Chang  argue  that  we  can 
also  scale  u(*)  to  be  approximatley  linear  in  terms  of  goods  in  gen¬ 
eral.  Under  the  standard  von  Neumann-Morgenstern  axioms  for  expected 
utility  maximization  we  can  now  formulate  the  problem  as 


(1.1)  v(An,Rn)  =  Max  E^,c  /e-Pt [u(c.  )dt  +  PdA(t)| 

U  U  A,c  A0 ’  u  0  t 


subject  to: 


(1.2)  R(t)  =  /[- c  ds  +  cUr^(s) j  >0Yt  >  0 
0  3 


R(o)  =  R0;  A(0)  =  A0  , 


where  denotes  expectation  under  the  exploration  policy  A  and 

the  consumption  policy  c,  with  initial  resource  stocks  x  and  initial 
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unexplored  land  y.  Note  that  the  second  (Stieltjes)  integral  of  (l.l) 
is  negative  and  decreasing,  since  A  is  decreasing. 

In  each  of  the  above  cited  papers  resources  are  located  at 
discrete  points  (in  "lumps'')  according  to  a  Poisson  distribution. 

See  Feller  [19681  page  159  for  a  good  interpretation  of  Poisson 
distributions  of  points  in  Euclidean  space.  Thus,  for  example,  the 
exploration  policy  A(t)  =  Aq  -  t  would  generate  new  discoveries  at 
Poisson  arrival  times. 

Both  Arrow  and  Chang  [1978]  and  Derzko  and  Sethi  [1981]  assume 
that  each  discovery  yields  a  fixed  amount  of  the  resource,  while  H-C-K 
allow  the  size  of  the  discovery  to  be  random  according  to  some  general 
probability  distribution  with  positive  support. 

One  of  the  purposes  of  thi3  paper  is  to  reformulate  as  a 
controlled  diffusion  process  and  obtain  the  insights  which  this  allows. 
To  do  this  requires  a  careful  specification  of  what  "instantaneous 
control"  means  for  Arrow's  model.  This  also  serves  to  place  a  more 
rigorous  foundation  under  the  previously  cited  studies.  For  the  remain¬ 
der  of  this  section,  however,  we  will  proceed  at  a  heuristic  level  to 
give  the  basic  flavor  of  stochastic  dynamic  programming  applied  to 
Arrow's  model.  All  of  the  above  mentioned  papers  proceed  roughly  along 
the  following  lines. 

Given  the  (i.i.d.)  probability  assumptions,  the  usual  line  of 
attack  of  Markovian  stochastic  dynamic  programming  can  be  taken.  Under 
an  optimal  policy,  at  any  time  t,  V(At,Rt)  from  (l.l)  (reinitialized  at 
time  t)  can  be  equated  with  the  expected  accumulation  of  utility  over 
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any  interval  of  time  At,  plus  the  expected  discounted  value  of  V  at 
time  t  +  At.  There  are  some  formal  mathematical  difficulties  (which 
are  not  resolved  in  any  of  the  previous  studies)  in  taking  the  limit  of 
this  sum  divided  by  At  as  At  +  0.  These  difficulties  are  resolved  by 
the  technique  introduced  in  Section  2,  but  for  the  present  we  will 
proceed  loosely.  First  assume  that  A  is  purely  discontinuous  with 
"very  small"  jumps.  Let  AA  denote  the  jump  of  A  at  time  t  (and 

ti 

bear  in  mind  in  the  following  caluclations  that  Aa  <  0).  We  then  have 

x 

from  (l.l)  the  approximation  for  "small"  At 


(1.3)  V(A.  ,R.  )  ~  Max  {u(c.  )At  +  PAA.  +  e~p(t+At)E lv(A.  +  AA.  ,R  +  AR  ) ]  } 
ct,OAt 

~  Max  {u(c  )At  +  PAA  +  (l  -  pAt)V(A  ,R  )  -  V  c  At 
=  .  t,  x  x  x  n  z 

ct*  \ 

♦  VAAAt  -  AV(A,R)AAt}  . 


where 


va(a,r) 


3V(A,R) 

3A 


-  3V  ( A ,  R ) 
R  '  3R 


and 


AV  ( A ,  R )  =  lim  E  [XljbJLlJgL 

A  A  tO  " 


that  is,  AV  is  the  "expected  rate"  of  improvement  in  V  induced  by 
additions  to  stockpiles  R  resulting  from  exploration  at  a  unit  rate 
(holding  A  constant).  For  example,  under  the  Poisson  distributional 
assumption,  AV(a,R)  =  B[V(a,  R  +  l)  -  V(A,R)1,  where  B  is  the  para¬ 
meter  of  the  Poisson  distribution. 


t 
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Simplifying,  we  have  the  Bellnan  equation: 

(l.U)  pVAt  -  Max  {u(c„  )At  -  VncJt  +  [-  AV  +  P  +  V.  j  AA.  }  . 

9  %  “t  A*t 

C  AA 

t  t 

From  the  first  order  conditions  for  optimality  in  (1.1»)  we  have 

(1.5)  c*  *  C(VR)  s  u’_1(VR)  , 

where  u'-1  is  the  functional  inverse  of  u',  well  defined  since  u  is 
strictly  concave.  Furthermore, 

(1.6)  AA*  =  0  if  AV  -  P  -  V.  <  0  , 

A 

and  otherwise: 

(1.7)  AA*  =  sup  {AA  <  0:  AV(A  +  AA,R  +  AR)  -  P  -  V  (A  +  AA.R  +  AR)  <  0 

or  +  AA  =  0} 

The  recipe  (1.6)-(1.7)  for  AA*  obviously  needs  some  interpretation. 

The  exploration  policy  A  is  apparently  "hang-bang."  If  the  marginal 
expected  return  to  exploring,  AV  -  P  -  V^,  is  negative,  zero  exploration 
is  optimal  (1.6).  Otherwise,  land  should  be  explored  instantaneously 
"until"  either  the  random  amount  of  resources  extracted,  AR  in  (1.7), 
is  sufficient  to  push  the  marginal  return  on  exploration  below  zero,  or 
until  the  remaining  land  is  exhausted.  Note  that  AA*  is  thus  a  random 
"probing  variable."  Instantaneous  control  for  this  problem  is  not 
merely  a  case  of  "infinite  exploration  rate,"  but  also  involves  sequen¬ 
tial  decisions  and  realizations  of  random  discoveries  all  at  a  single 
point  of  time!  (See  Section  2.) 
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Assuming  all  of  the  above  can  be  made  rigorous,  the  state  space 
{(A,R)  £  ]R^}  can  be  divided  into  two  regions,  no  exploration  and 
instantaneous  exploration,  as  illustrated  in  Figure  1.1. 

Figure  1.1  Control  Regions  in  the  State  Space 


In  region  Dq  (no  exploration)  we  can  eliminate  the  last  terra  of 
the  Bellman  equation  (l.U)  since  AA  -  0  and  divide  through  by  At, 
obtaining  the  following  differential  equation  for  V  after  substituting 
from  (1.5): 

(1.8)  PV(A,R)  =  uIC(VR)]  -  VRC(VR)  ,  (A ,R )  G  DQ  . 

Equation  (1.8)  is  actually  a  first  order  differential  equation  for  V 
as  a  function  of  R,  of  the  form: 

(1.9)  V(A,R)  *  V(R  +  RE(A))  , 

where  Rg(A)  is  a  constant  representing  the  "resource  equivalent"  of 
the  remainng  unexplored  land  A.  For  the  case  u(c)  *  In  (c),  W(»)  has 
the  solution 

(1.10)  W(x)  »  ^  lln(px)  -  l)  , 


which  implies,  using  (1.5), 


-7- 


(1.11)  C»  (A,R)  =  p[R  +  Rg(A) ]  . 

Similar  solutions  are  available  for  the  case  of  power  utilities  and,  in 
general, 

(1-12)  c*  (A,R)  =  ^  , 

RR 

which  is  the  classical  Hotelling  [1931 1  result  in  the  deterministic  case 
(A  =  RE(A)  =0). 

Turning  to  the  instantaneous  exploration  region  Dw,  we  can  divide 
the  Bellman  equation  (l.U)  through  by  AA,  eliminating  terms  in  At 
since  "instantaneous”  of  course  means  At/AA  ■  0.  This  leaves  the 
inequality,  using  (l.6)-(l.7), 

(1.13)  AV  -  P  -  VA  >  0  (A,R)  G  D,,,  . 

Derzko  and  Sethi  [l98l]  give  a  common  sense  argument  that  (1.13)  nmt  in 
fact  hold  with  equality  in  D^,  while  H-C-K  give  a  more  formal  argument 
based  on  the  transition  probability  governing  movement  in  Dw. 

Thus  the  equation 

(1.1U)  AV(A,R)  -  P  -  Va(A,R)  *  0  (A,R)  G  D„ 

along  with  the  (1.8)  and  the  obvious  boundary  conditions  constitute  the 
information  available  to  determine  a  solution  for  V(a,R).  The  key 
elements  of  the  solution  are  the  two  functions:  Rg(A),  the  boundary 
curve  separating  Dq  from  D„;  and  Re(A),  the  "resource  equivalents" 


function. 


The  papers  by  H-C-K  and  Derzko  and  Sethi  [l98ll  both  contain 
numerical  approximations  for  Rg( • ).  Their  results  are  similar  although 
obtained  by  quite  different  routes.  H-C-K  also  produced  numerical 
approximations  for  R„( • )  and  extended  the  price  dynamics  analysis  of 

£j 

Arrow  and  Chang  [19781. 

Sundaresan  [1983!  includes  a  survey  of  research  in  optimal  extrac¬ 
tion  of  nonrenewable  resources  under  uncertainty. 

2.  The  Control  Clock 

Before  proceeding  to  a  diffusion  formulation  of  Arrow's  model,  it 
is  important  to  place  the  basic  model  on  a  rigorous  foundation  by  for¬ 
malizing  the  notion  of  instantantaneous  control  suggested  in  Section  1. 

It  is  inherent  in  Arrow's  model  that,  during  exploration,  deci¬ 
sions  of  whether  or  not  to  continue  exploring  may  be  made  successively 
at  a  single  point  of  time.  This  is  a  much  stronger  interpretation  of 
"instantaneous  control"  than  has  usually  been  considered  (e.g.  Harrison 
and  Taksar  [1982]).  Here  one  may  explore  a  certain  area  of  land,  re¬ 
evaluate  the  situation  based  on  new  discovery  information,  and  perhaps 
continue  exploring,  all  at  the  same  point  of  time.  It  is  implicit  that 
there  are  two  "clocks"  running,  one  describing  the  passage  of  "real" 
time,  the  other  tracking  the  running  down  of  unexplored  land.  During 
instantaneous  exploration  the  first  clock  is  not  running,  while  the 
second  records  the  using  up  of  unexplored  land  and  generates  information 
about  the  increnv  'ts  of  the  liscoveries  proceeds,  N^.  For  a  well  de¬ 
fined  control  prob’  ,  however,  the  control  policy  should  be  formally 
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adapted  to  some  information  structural  F  *  {Ft,x  >  0}  vhere,  roughly 
speaking,  FT  describes  the  information  known  at  "time  x"  on  some 
"control  clock.”  A  convenient  way  to  set  up  the  control  clock  is  to  let 
"control  time"  be  the  sum  of  real  time  and  cumulative  land  explored. 
Accordingly,  ve  relate  t(x),  (real  time  at  t  on  the  control  clock), 
and  L(x)  =  Aq  -  A(x)  (land  explored)  by: 

T 

(2.1)  L(x)  -  [X  ds  >  0 

0  3 

x 

(2.2)  t( x )  =  /(I  -  X  )ds  >  0 

0  8 


so  that 


(2.3) 


x  *  L(r)  +  t(r)  >  0  , 


where  X^  G  lO,l]  is  the  "exploration  intensity"  (a  control  process). 
When  Xt  =  l,  real  time  is  standing  still  (2.2)  while  exploration  takes 
place  at  full  intensity  (instantaneously  in  real  time).  When  X^  =  0 
no  exploration  is  occurring.  Finally  X^  e  [0 , 1 1  implies  that  real 
time  and  exploration  are  running  simultaneously. 

With  this  control  clock,  X(x),  A(x),  t(x),  R(x),  and  c(t(x))  can 
all  be  formally  modeled  as  stochastic  processes  adapted  to  some  exogen¬ 
ous  information  structure.  If  the  notion  that  exploration  can  actually 
occur  instantaneously  is  rejected,  the  exploration  control  X  can  be 
restricted  to  a  compact  subinterval  of  |0,l). 

It  is  a  difficult  task  indeed  to  formalize  the  analysis  in  Section 


1  (and  in  the  earlier  papers)  without  a  device  similar  to  this  time 
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change.  For  example,  it  was  assumed  throughout  the  derivation  of  the 
Bellman  equation  that  AA  is  "very  small,"  whereas  the  actual  policy 
involves  considerable  jumps  in  certain  regions  of  the  state  space. 

3.  A  Diffusion  Formulation  of  Arrow's  Model 

In  this  section  we  reformulate  Arrow's  model  using  a  diffusion 
idealization  of  the  discoveries  process  N^.  Although  this  does  not 
inq>rove  on  the  Poisson  distributional  assumption  by  yielding  an  analyti¬ 
cal  solution  (as  one  might  have  hoped),  it  does  promote  a  somewhat 
clearer  understanding  of  certain  points.  It  also  establishes  the 
robustness  of  certain  of  Arrow  and  Chang’s  [1978]  original  results  under 
quite  different  probabilistic  assumptions. 

As  a  preliminary,  let  B  -  (3^:  T  >  0;  £2,F,P)  denote  a  Standard 
Brownian  Motion  on  the  probability  space  (il,F,P). 

We  begin  by  suggesting  that  the  exploration  activity  requires 
stochastic  inputs  I  =  (i^;  t  >  0)  of  the  resource  itself,  where  1^ 
is  the  amount  of  the  resource  required  to  explore  the  first  Lt  units 
of  land  area.  The  stochastic  process  0  =  (0r,x  >  0)  similarly  des¬ 
cribes  gross  discoveries  (output)  of  the  resource. 

If  the  input  and  output  processes  I  and  0  have  i.i.d. 
increments  over  each  successive  section  of  land  of  fixed  area,  then  a 
natural  idealization  of  the  net  discoveries  process  N  5  0  -  I  is  the 
controlled  diffusion  process: 

x  x 

N  =  /X  [yds  +  odB  | 

T  o  s  s 


(3.1) 
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for  some  positive  constants  M  and  o.  See  Harrison  [1981]  for  the 

arguments  leading  to  this  idealization  of  a  production  process. 

From  (3.1)  we  see  that  N*  has  drift  X^p  and  "instantaneous 
2  2 

variance"  X^o  which  depend  on  the  exploration  control  X.  In 
effect,  X  "speeds  up"  or  "slows  down"  the  (p,o)  Brownian  motion 
discoveries  process  which  would  result  if  exploration  were  carried  out 
at  a  uniform  rate. 

control  problem  (l.l)-(l.2)-(l.3)  is  unchanged,  but  can  now  be 
set  UD  in  "control  time,"  using  (2.1)  and  (2.2),  as: 

p 

vind  V:  JR'-  ♦  ?  and  non-anticipating  controls  X  and 
cT  =  c't(x) )  satisfying 

GD 

(3.2)  V(A,R)  =  Max  E*C,{/e_Pt(  t)  [u(c  )  (l  -  X  )  -  PX  Jdx} 

XC  0  T  T  T 

where 

(3.3)  dt( t )  =  (1  -  xT)dt  ,  t(o)  =0  ,  xt  £  [0,1) 

(?.^)  d?T  =  d"*  -  cT(l  -  Xx)dT  ,  Rt  >  0 

(3.5)  dtf*  =  X  udT  +  X  odB 

T  T  T  X 

(3.o)  dAT  =  -X^dx  ,  AT  >  0 

To  guarantee  that  the  above  integrals  are  well-defined  (with  V 
permitted  to  take  the  value  -") ,  we  assume  that  c  G  [0,c]  for  some 
(large)  constant  c.  The  conditions  on  u  given  in  Section  1  then 
place  an  upper  bound  on  u(c  ). 
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We  will  proceed  on  the  basis  that  a  solution  to  (3.2)  exists  with 

2 

VR  •  VA  •  “4  VRR  5  (A-R) 

existing  and  bounded  (at  least  for  A  >  0,  R  >  0).  Using  Ito's  lemma, 

Xc 

the  infinitesimal  generator  L  corresponding  to  the  transition  semi¬ 
group  for  the  controlled  process  NA  yields  (using  (3.M,  (3.5)  and 

(3.6) ): 

2 

(3.7)  LXcV(A,R)  =  -AVa  +  VR[pA  -  c(l  -  A) ]  +  A2  VRR  . 

Then  applying  the  strong  Markov  property  of  B  and  relation  (3.3),  we 
derive  the  Bellman  equation  from  (3.2): 

(3.8)  V  -  Max  {ll  -  p(l  -  A) }V  ♦  u(c)(l  -  A)  -  PA  +  LAcv} 

Ac 

subject  to:  c>0  ,  A  £  [0,1 )  . 

(See,  for  example,  Fleming  and  Rishel  [19751  Chapter  VI  for  details.) 
Note  that  with  the  advantages  of  a  "control  clock"  and  the  continuity  of 
diffusion  processes,  the  derivation  of  the  Bellman  equation  is  quick  and 
without  approximation. 

Since  u(*)  is  strictly  concave  it  is  easy  to  deduce  from  (3.2) 
that  VRR  <0.  We  can  then  define 

(3.9)  A(A,R)  5  -  [pV  -  u(c(VR))  -  P  -  VA  +  VR(w  +  c(VR))J  . 

V__  o 
RR 


I 
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The  first  order  necessary  conditions  for  optimality  in  (3.8)  then 
generate  the  following  solutiuon  for  X»(A,R)  and  c*(A,R),  the  optiml 
exploration  and  consumption  controls: 

A 

X(A,R)  >  1  =*  X*(A,R)  *  1;  c*(A,R)  arbitrary 

(3.10)  0  <  X(A,R)  <  1  =*  X*(a,R)  *  X(A,R);  c*(A,R)  =  C(VR) 

X(A,R)  <  0  =»  X»(A,R)  *  0;  c»(A,R)  *  C(VR)  . 

Combining  (3.7),  (3.8),  and  (3.10)  and  simplifying,  we  have  the  final 
form  of  the  Bellman  equation: 

p  „2 

(3.11)  (1  -  x»)l-c*VR  -  pV  +  u(c*)]  +  X»(uVR  -  Vft  -  P)  +  X*^  — ^  VRR  »  0 

Note  that  (3.1l)  is  a  partial  differential  equation  for  V  which 
must  hold  throughout  the  state  space.  Just  as  in  Section  1,  however,  we 
can  simplify  the  Bellman  equation  by  dividing  up  the  state-space,  this 
time  into  the  three  regions: 

Dq  *  {(A,R)  6  ]R+:  X(a,R)  <  0)  (consunqytion  only) 

2  A 

D  *  {(A,R)  e  ]R+.  X(a,R)  €  (0,1))  (simultaneous  consumption 

and  exploration) 

2  * 

■  {(A,R)  €  ]R  :  X(a,R)  >1}  (instantaneous  exploration) 

The  intermediate  region  Dit  where  exploration  and  consumption 
occur  simultaneously  and  continuously,  is  also  feasible  for  the  formula¬ 
tion  in  Section  1,  but  can  be  rejected  there  as  sub-optimal  from  the 
pure  discontinuity  of  N^.  In  the  diffusion  case,  however,  we  see  from 
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(3.9)  that  is  null  only  if  there  is  a  solution  V(A,R)  for  the 

Bellman  equation  (3.11)  satisfying: 

(3.12)  PV  =  u((VR))  -  P  -  VA  +  VR(p  +  C(VR))  =  0 

with 


X»(A,R)  *  X(A,R)  =  0 


and 

A 

c*(A,R)  *  C(V  )  . 

In  the  case  of  logarithmic  utility,  u(c)  =  ln(c),  the  following 
specification  for  V  satisfies  (3.1l)  and,  if  ?  =  0,  (3.12): 

V(A,R)  *  i  ln(p(R  +  UA))  +  . 

We  feel  this  to  be  somewhat  of  a  quirk,  however,  related  to  the  very 

special  properties  of  logarithmic  utility  (c(V  )V  =  l).  In  general 

n  n 

the  solution  is  enriched  by  situations  in  which  it  is  optimal  to  consume 
and  explore  simultaneously. 

In  Dq,  the  "consume  only"  region,  the  same  ODE  (1.9)  found  in 
Section  1  is  obtained  from  the  Bellman  equation  (3.1l)  for  X*  =  0. 

That  is,  it  is  again  optimal  to  consume  in  DQ  according  to  the 
"Hotelling  rule"  (1.12)  until  the  reduction  in  stocks  induces  further 
exploration. 


I 
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Finally  in  D„,  where  X*  *  1,  we  obtain  from  (3.11) 

(3.1k)  „,R.4vm-vA-P-o  . 

Note  that  (3. I1*)  is  the  direct  analogue  of  (1.1*0,  which  was  obtained  by 
a  much  more  tortuous  route. 

In  fact  (3.1*0  and  (1.1*0  can  be  shown  to  be  formally  equivalent 
by  applying  Ito's  lemma  to  the  definition  of  AV  in  (1.3). 

In  the  diffusion  case  ( 3 . 11* )  has  an  interesting  interpretation  as 
the  Kolmogorov  Backward  equation  governing  the  transition  density  of 
V  inside  D^.  Define  the  stopping  time 

T  =  inf  (t  >  0:  (At,Rt)  Dj 

and  consider  the  stopped  process  v^atAT’RtaT^*  Since  X  =  1  in  D». 

p 

R  moves  in  Dw  simply  as  a  (y,a  )  Brownian  motion,  and  dA^  =  dT 
from  (2.1).  For  any  stopping  time  s  <  T,  by  Bellman’s  Principle  of 
Optimality  we  have,  for  (A,R)  GD# 

(3.15)  V(A,R)  *  E(V(As,Rs)  -  P(Ag  -  at)|At  *  A,RT  =  R) 

*  E(V(A  -  s  +  T,R  )  -  P(s  -  t)|R  *  Ri  . 

3  i 

In  particular,  for  s  *  T,  (3.15)  says  that  the  current  value  of  V  at 
a  point  in  Dw  is  the  expected  value  of  V  at  the  exit  point 
(A^,R^,),  less  the  expected  exploration  cost  to  "get  out,"  E|P(T  -  t)J. 
Now  define  Y(s,t,r)  by  equating  it  with  the  right-hand  aide  of 

(3.15) .  Then  (3.1*0  is  formally  the  backward  equation  for  Y(s,t,R), 

p 

since  Rt  is  a  (ti,o  )  Brownian  motion  in  D,,.  Thus,  since 


I 
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Y(t,t,r)  =  V(A,R)  for  (A,R)  €  D^,  (3.14)  is  the  partial  differential 
equation  giving  the  transition  density  of  V,  treating  land  as  "time." 

4.  Mapping  from  Control  Time  Back  to  Real  Time 

In  the  last  section  ve  characterized  the  optimal  controls  X* 
and  c*  as  feedback  policies  on  the  current  state  variables,  unexplored 
land  (A)  and  stockpiles  of  the  resource  (R).  Given  the  simple 
relationship  (2.1)  between  "real  time"  and  "control  time,"  it  is  simple 
to  map  these  policies,  which  apply  in  control  time,  back  into  real  time. 

Since  the  consumption  policy  cT  =  c(t(t))  was  formulated  in  real 
time  to  begin  with,  we  have  the  solution 

(4.1)  e*(t)  «  c(VR(At,Rt))  , 

which  holds  everywhere  (since  the  consumption  policy  is  irrelevant  in 

DJ- 

The  exploration  policy  must  be  characterized  separately  in  each  of 
the  regions  from  (3.6)  and  (3.10).  In  D^,  it  is  impossible  to  express 
the  policy  in  real  time  except  to  say  that  it  is  optimal  to  explore 
"instantaneously"  to  the  boundary  of  Dw.  In  some  of  the  previously 
cited  studies  it  has  been  overlooked  that  this  may  entail  exploration  of 
all  remaining  land  without  ever  reaching  the  boundary  curve  Rg( • ) 
illustrated  in  Figure  1.  In  the  case  of  diffusion  discoveries,  it  may 
even  occur  that  instantaneous  exploration  will  exhaust  all  stocks  R 
before  the  remaining  land  is  all  explored.  Of  course  the  probability  of  such 
an  occurrence  will  be  small  given  reasonable  choices  for  u  and  o. 


I 
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(U.2) 


(U.3) 

where 


In  Dq,  by  definition,  the  optimal  exploration  policy  A*  is 
A*(t)  5  =  0  . 


Finally,  in  the  intermediate  region  we  have 


A#(t) 


- 

i  - 


1  is  the  function  defined  in  (3.9). 
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Foot  notes 


1/  Formally,  we  should  preface  these  remarks  by  a  statement  that 

(Q,F,P)  is  a  probability  space  describing  uncertainty  in  Arrow's 
model.  Then  F  is  a  filtration .  a  right-continuous  increasing 
family  of  sub-tribes  of  F,  with  F  =  U F  .  See,  for  example 
Meyer  (196? 1,  p.  65.  ‘  *  T 


-19- 


References 


Arrow,  K.  J.  1 1977 1 »  "Optimal  Pricing,  Use,  and  Exploration  of  Uncertain 
Natural  Resource  Stocks,"  (presented  at  the  Conference  on  Natural 
Resource  Pricing,  Trail  Lake,  Wyoming,  August  1977). 

Arrow,  K.  J.  and  S.  Chang  1 1980 ) ,  "Optimal  Pricing,  Use  and  Exploration 
of  Uncertain  Natural  Resource  Stocks,"  in  Pan-Tai  Liu  (Ed.) 

Dynamic  Optimization  in  Mathematical  Economics.  ( Plenum  Press , 

New  York). 

Derzko,  N.  A.  and  S.  P.  Sethi  [1981I,  "Optimal  Exploration  and  Consump¬ 
tion  cf  a  Natural  Resource — Stochastic  Case,"  International  Jour¬ 
nal  of  Policy  Analysis  and  Information  Systems,  Vol.  5,  No.  3,  pp. 
195-200. 

Feller,  W.  I1968] ,  An  Introduction  to  Probability  Theory  and  Its  Appli¬ 
cations  (Third  Edition).  John  Wiley  and  Sons,  New  York.  ’ 

Fleming,  W.  H.  and  V.  R.  Rishel  1 1975 ] deterministic  and  Stochastic 
Optimal  Control.  Springer-Verlag,  Berlin. 

Hagan,  P.  S.,  R.  E.  Caflisch,  and  J.  B.  Keller  [1981],  "Arrow's  Model  of 
Optimal  Pricing,  Use  and  Exploration  of  Uncertain  Natural 
Resources,"  Submitted  to  Econometrics. 

Harrison,  J.  M.  (19821 ,  "A  Diffusion  Model  of  Inventory  and  Production 
Control,"  Technical  Report,  Department  of  Operations  Research, 
Stanford  University. 

Harrison,  J.  M.  and  M.  I.  Taksar  [19831 ,  "Instantaneous  Control  of 

Brownian  Motion,"  to  appear  in  Mathematics  of  Operations  Research. 

Hotelling,  H.  1 1931 ] ,  "The  Economics  of  Exhaustible  Resources,"  Journal 
of  Political  Econony,  Vol.  39,  pp.  137-175. 

Karlin,  S.  and  H.  M.  Taylor  I1981 1  ,  A  Second  Course  in  Stochastic  Pro¬ 
cesses.  Academic  Press,  New  York. 

Meyer,  P.  A.  (19671,  Probability  and  Potentials.  Blaisdell  Publishing 
Co.,  Waltham,  Mass. 

Sundaresan,  M.  I1983I »  "Optimal  Extraction  Policies  and  Equilibrium  Spot 
and  Futures  Prices  of  Nonrenewable  Resources,"  Graduate  School  of 
Business,  Columbia  University. 


mill'd i  r“r  iPi|l|i,\  i"  >  P'"-  <i"i""hm  "",'s  S3HHS  sihj  si  sj 


I 


376.  "Necessary  and  Sufficient  Conditions  for  Single-Peakedness  Along  a 
Linearly  Ordered  Set  of  Policy  Alternatives"  by  P.J.  Coughlin  and 
M.J.  Hinich. 

377.  "The  Role  of  Reputation  in  a  Regeated  Agency  Problem  Involving  Information 
Transmission"  by  W.  P.  Rogerson. 

378.  "Unemployment  Equilibrium  with  Stochastic  Rationing  of  Supplies"  by 
Ho-mou  Wu. 

379.  "Optimal  Price  and  Income  Regulation  Under  Uncertainty  in  the  Model  with 
One  Producer"  by  M.  I.  Taksar. 

380.  "On  the  NTU  Value"  by  Robert  J.  Aumann. 

381.  "Best  Invariant  Estimation  of  a  Direction  Parameter  with  Application  to 
Linear  Functional  Relationships  and  Factor  Analysis"  by  T.  W.  Anderson, 

C.  Stein  and  A.  Zaman. 

382.  "Informational  Equilibrium"  by  Robert  East. 

383.  "Cooperative  Oligopoly  Equilibrium"  by  Mordecai  Kurz. 

38U.  "Reputation  and  Product  Quality"  by  William  P.  Rogerson. 

385.  "Auditing:  Perspectives  from  Multiperson  Decision  Theory"  By  Robert 
Wilson. 

386.  "Capacity  Pricing"  by  Oren,  Smith  and  Wilson. 

387.  "Consequentialism  and  Rationality  in  Dynamic  Choice  Under  Uncertainty" 
by  P.J.  Hammond. 

388.  "The  Structure  of  Wage  Contracts  in  Repeated  Agency  Models"  by  W.  P. 
Rogerson . 

389.  "1982  Abraham  Wald  Memorial  Lectures,  Estimating  Linear  Statistical 
Relationships  by  T.W.  Anderson. 

390.  "Aggregates,  Activities  and  Overheads"  by  W.M.  Gorman. 

391.  "Double  Auctions"  by  Robert  Wilson. 

392.  "Efficiency  and  Fairness  in  the  Design  of  Bilateral  Contracts"  by  S. 
Honkapohja. 

393.  "Diagonality  of  Cost  Allocation  Prices"  by  L.J.  Mirman  and  A.  Neyman 

39U.  "General  Asset  Markets,  Private  Capital  Formation,  and  the  Existence 
of  Temporary  Walrasian  Equilibrium"  by  P.J.  Hammond 

395*  "Asymptotic  Normality  of  the  Censored  and  Truncated  Least  Absolute 
Deviations  Estimators"  by  J.L.  Povell 

396.  "Dominance-Solvability  and  Cournot  Stability"  by  Herve  Moulin 

397.  "Managerial  Incentives,  Investment  and  Aggregate  Implications"  by 
B.  Holmstrom  and  L.  Weiss 


398  "Generalizations  of  the  Censored  and  Truncated  Least  Absolute  Deviations 
Estimators"  by  J.L.  Powell- 

399-  "Behavior  Under  Uncertainty  and  its  Implications  for  Policy"  by  K.J.  Arrow. 

UOO.  "Third-Order  Efficiency  of  the  Extended  Maximum  Likelihood  Estimators  in 
a  Simultaneous  Equation  System"  by  K.  Takeuchi  and  K.  Morimune. 

1*01.  "Short-Run  Analysis  of  Fiscal  Policy  in  a  Simple  Perfect  Foresight  Model" 
by  K.  Judd. 

U02.  "Estimation  of  Failure  Rate  From  A  Complete  Record  of  Failures  and  a 
Partial  Record  of  Non-Failures"  by  K.  Suzuki. 

1*03-  "Applications  of  Semi-Regenerative  Theory  to  Computations  of  Stationary 
Distributions  of  Markov  Chains"  by  W.K.  Grassmann  and  M.I.  Taksar. 

Uol*.  "On  the  Optimality  of  Individual  Behavior  in  First  Come  Last  Served  Queues 
With  Preemption  and  Balking"  by  Refael  Hassin. 

1*05-  "Entry  with  Exit:  An  Extensive  Form  Treatment  of  Predation  with  Financial 
Constraints"  by  J.P.  Benoit. 

1*06.  "Search  Among  Queues"  by  A.  Glazer  and  R.  Hassin 

1*07-  "The  Space  of  Polynomials  in  Measures  is  Internal"  by  J.  Reichers  and  Y. 
Tauman. 

I*08.  "Planning  Under  Incomplete  Information  and  the  Ratchet  Effect"  by 
X.  Freixas,  R.  Guesnerie  and  J.  Tirole. 

409.  "A  Theory  of  Dynamic  Oligopoly,  I:  Overview  and  Quantity  Competition  with 
Large  Fixed  Costs"  by  Eric  Maskin  and  Jean  Tirole. 

410.  "Approximate  Measures  of  the  Social  Welfare  Benefits  of  Labor  Projects" 
by  Peter  Hammond 

1*11.  "Transversality  Conditions  for  Some  Infinite  Horizon  Discrete  Time  Optimiza¬ 
tion  Problems"  by  Ivar  Ekeland  and  Jose  A.  Scheinkman. 

1*12.  "Asymptotic  Efficiency  of  the  Partial  Likelihood  Estimator  in  the  Proportional 
Hazard  Model"  by  Aaron  K.  Hahn. 

1*13.  "A  Monte  Carlo  Comparison  of  Estimators  for  Censored  Regression  Models"  by 
Harry  J.  Paarsch. 

1*1 1* .  "Instrumental-Variable  Estimation  of  an  Error-Components  Model"  by  Takeshi 
Amemiya  and  Thomas  E.  MaCurdy. 

1*15-  "An  Axiomatizatlon  of  the  Non-Trans ferable  Utility  Value"  by  Robert  J.  Aumann. 

1*16.  "Diffusion  Approximation  in  Arrow's  Mode]  of  Exhaustable  Resources"  by 
Darrell  Duffie  aril  Michael  Taksar. 


